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ABSTRACT 



Library 
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The usefulness of random walks in mathematical modeling is often 
overshadowed by the problems that confront both students and instructors 
of probability. The counter-intuitive conclusions which arise produce 
both ambiguities and misunderstandings. 

In this thesis, the techniques of computer simulation have been 
combined with the visual appeal of interactive graphic displays to de- 
velop a simulation of random walks. This simulation features interactive 
routines which are easy to use and take advantage of the insight and 
visual capabilities of the user to build an intuitive background of the 
subject matter. Statistics whose empirical distributions are 
asymptotically Arc Sine and Normal plus the gamblers ruin problem are 
displayed under various experimental conditions which the user designs. 

This simulation is appropriate for use both by instructors to com- 
plement their classroom presentations and by students to enhance their 
understanding of the theory. 
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I. INTRODUCTION 



Some of the simplest and most versatile of the stochastic processes 
are random walks. A random walk, as defined by Barr and Zehna (Ref. 1), 
is simply a point moving along the real number line in such a way that 
at each time t(t =0, 1, 2, ...), the point is an integer on the line. 

When the coordinates of the point are (t, 0), a "equalization 11 or a 
"return to equilibrium" is said to have occurred. 

Random walks are one-dimensional Markov chains and are used to model 
a wide range of phenomena, all characterized by random fluctuations. 

Such uses include exact models of simple coin-tossing games and first 
approximations to chance-dependent processes in Physics, Economics, and 
learning theory. Also included are the probabilistic development of 
Galton f s rank order test, tests of the Kolmogorov- Smirnov type and 
sequential sampling. 

Despite the simplicity of the structure of random walks and their 
wide range of uses, classroom presentations of the theory of random 
walks pose problems for both instructors and students of probability. 

These problems arise from the fact that although the basic premises of 
random walks can be developed quite simply from elementary combinatorial 
techniques, the conclusions encountered are often quite counter-intuitive 
in nature. 

For example, if we visualize a random walk in terms of a simple coin- 
tossing game, the path followed by the random walk represents the 
cumulative number of heads minus tails at any point in time during the 
game. With probability one-half, no equalization will occur in the 
second half of the game regardless of the length of the game. In fact, 
it is quite likely that the winning player will be ahead for the entire 
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game. Also, the maximum accumulated gain, which is the highest value 
taken on by the cummulative number of heads minus tails during the game, 
is more likely to be attained at the very beginning or very end of the 
game. 

The above aspects of random walks, exemplified by the simple coin- 
tossing game, are counter-intuitive to beginning probability students 
because of the preconceived notions concerning chance fluctuations and 
implications of the law of large numbers. Students often mistakingly 
assume that one coin tossed many times will yield the same statistical 
characteristics as an equal number of coins tossed once. The total of 
the 'heads minus tails is the same in both instances. It is the manner 
in which the single coin tossed many times arrives at this total that is 
the source of the student’s misunderstanding. 

A computerized simulation of random walks, combined with the capa- 
bility to graphically display the results of the simulation, provides a 
very practical instructional device for dynamically representing some of 
the more subtle aspects of random walk theory. Interactive routines 
which feature graphic displays facilitate the development of student 
insight and comprehension by permitting the student to repeat specific 
simulations, vary parameters and to observe the effect on resulting 
displays. This helps to solidify and re-affirm his convictions on the 
subject matter. Instructors can use such a device equally well to com- 
plement their classroom presentations and to stimulate student interest 
in the topics by simulating and dynamically displaying examples of the 
material they are presenting. 
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II. DESCRIPTION OF A RANDOM WALK 



A simple and intuitive characterization of a random walk is a coin- 
tossing game in which the primary statistic of interest is the cumulative 
number of heads minus tails. This total is denoted by x which repre- 
sents the value of the cumulative number of heads minus tails at the 
th 

time of the n toss of the coin. The event of a head occuring on a 
particular toss of the coin results in a unit increment to the cumulative 
number of heads minus tails. The probability of this event is called the 
success probability and is denoted by p. Decrements occur with proba- 
bility 1-p. Each change in the cumulative number of heads minus tails is 
the result of an independent trial with the success probability p. 

If the success probability p is one-half, the random walk is said 
to be symmetric, otherwise it is said to be an unsymmetric random walk. 
Random variables will be displayed and discussed in terms of this char- 
acterization. Figure 1, on Page 9 , depicts a representative random 
walk. In this diagram, the cumulative number of heads minus tails is 
three after the fifth toss of the coin. 

More formally stated, a random walk is defined as a polygonal line 
running from the origin to an arbitrary point (n,x) whose vertices have 
abscissas 0, 1, . ...n, and ordinates Sq, S^, ... S n and where Sq = 0, 

= ±1 and = x. For example, x represents the total number of 
+l*s minus the number of -I's. Referring to Figure 1, we see that at 



n from any given point. From the origin to an arbitrary point (n,x) 



the point (5,3), = 3. 

In the symmetric case, there are 2 n equally likely paths of length 




^ i n + x . 

= 0 when — ~ — is not an 
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integer, x < n. The probability of visiting an arbitrary point x at 

time n denoted by P is: 
y n,x 



P 

n,x 



= p{s = 

n 



x 



} - N 



n.x 




2 



-n 



(II. 1) 



n 




Figure 1. A Random Walk Illustrating 
the Ballot Theorem 

The basic premises from which the theory of random walks is de- 
veloped are the reflection principle, the ballot theorem, and the con- 
cept of a "return to equilibrium." The reflection principle involves a 
line known as the equilibrium axis. This line represents all points in 
the path of the random walk where the cumulative number of heads minus 
tails is equal to zero. The reflection principle states that the number 
of paths between any two points A and B lying above the equilibrium axis 
which either touch or cross the axis is equal to the total number of paths 
from A 1 to B, where A f is the reflection of A below the axis , for example, 
if A = (n,x), then A* = (n,-x). Figure 2 is a graphic representation of 
the reflection principle, with points C, D, and E lying on the equili- 
brium axis. 
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which never touch or cross the axis. Specifically, those are the paths 

for which S~ = 0, S, > 0, . . . S = x. The random walk illustrated in 

Figure 1 is of this type with n = 10 and x = 2. The total number of 

such paths is the difference between the total number of paths from the 

point (1,1) to the point (n,x) , which ^ x and the num ber of 

paths from (1,1) to (n,x) which touch or cross the equilibrium axis, 

which is ^ according to the reflection principle. The desired 

result is then N . , - N , which is equal to — N 

n-l,x-l n-l,x+l n n,x 

A "return to equilibrium", denoted by is said to have occurred 
at stage k if = 0 where k is necessarily an even number. Letting 
k = 2v , the probability of this event occurring is given by formula II. 1 
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r '■ ^ - 




with n = v and x = 0 and is equal to 



M- 



2v 




(II. 2) 



In figure 2 on the preceeding page. Points C, D, and E represent "returns 
to equilibrium" where k = 2v is equal to four, six and eight, respective- 
ly. 

The probability that no return to equilibruim occurs up to and in- 
cluding stage 2v is the same as the probability that a return to equili- 
brium occurs at stage 2v. Symbolically, 



P{S L 7^ 0, ... S 2v t 0} = P{s 2v = 0} = p, 



2v 



(II. 3) 



Paths which satisfy this criterion are either above or below the equili- 
brium axis from time 1 to 2v, originating at the origin. There are 

N 5v- 1 2r-l — v ’ ‘ *P a ths .which, are ^.positive and. occur . 

with probability y ^anH'- an * equivalent' number which are* "negative-. 

Therefore: ‘ v- *’ - - • ‘ ~ ... 



* J, 2v 



2(k) 



P 2v-l,l* 



(II. 4) 
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III. RANDOM VARIABLES OF INTEREST 



A. SYMMETRIC RANDOM WALKS 

In terms of a coin-tossing game, there are four random variables 
that illustrate symmetric random walk concepts and are suitable for sim- 
ulation and graphic display. These are the cumulative number of heads 
minus tails, the last time the cumulative number of heads minus tails is 
equal to zero in the game, the number of tosses during which the cumula- 
tive number of heads minus tails is positive, and the toss at which the 
cumulative number of heads minus tails first reaches its maximum value 
in the game. 

The cumulative number of heads minus tails is the amount by which 
one player is ahead and the other is behind in the game. Figure 3 il- 
lustrates this random variable. Illustrated ■ is a game of ten, coin. . 
tosses in tthich one player is ahead for the first seven-. tos*s5s\ -.The 
game is even after the eighth toss and the other player *is ahead for 
the remaining two tosses. 




last return to equilibrium (B) 
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The cumulative number of heads minus tails (Sn) is the sum of n 



independent and identically distributed random variables, X^, 

which can take the value of +1 or -1, each with probability P = 

The expected value of is then E(S n ) = (p-q)n = 0 and the variance of 
S is V(S ) = 4pqn = n. 

The second random variable is the ,f last return to equilibrium 11 or 
the last time the total is equal to zero in the game. This occurs at 
the toss where the cumulative number of heads minus tails is equal to 
zero for the last time in the game. The Point B in Figure 3 on the 
preceeding page is illustrative of the last return to equilibrium in 
that particular random walk. 

The probability that up to and including time 2n, the last return 
to equilibrium occurred at time 2k is denoted by 2n* This is 

equivalent to the probability that a return to equilibrium occurs at 
t'ime' 2k and there are no subsequent returns in the remaining 2n-2k ^ 

time units of the trial. Formulas II. 2 and. II. 4 show these probabilities 
to be equal to and p^ n 2 ^ respectively. Since the events are mutu- 

ally independent, the desired probability is the product of the two 
individual probabilities. Hence, the formula is 



a 2k,2n ^2k ^2n-2k’ 



(III. A. 1) 



The third random variable is the time (i.e. toss) during which the 
cumulative number of heads minus tails is positive. It is represented 
in Figure 3, Page 12, by the eight units between the origin and the 
Point *B, where a return to equilibrium occurs. In general, the pro- 
bability that up to and including toss 2n of the coin this total is 
positive for exactly 2k tosses, is denoted b^^ 2 n * 
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The ways in which the number of heads minus tails can remain positive 
occurs in two ways. First, the cumulative number of heads minus tails 
may be positive for the first 2r tosses of the coin (r < k) and positive 
for exactly 2k-2r of the remaining 2n-2r tosses. Figure 4 illustrates 
an occurrance of this type. 




2r tosses and for exactly 2k-2r of the remaining 
2n-2r tosses. 

This 'event requires that the first return' to equilibrium, denoted by 

. . .. .. . . * , 

^2r ,OCCurs ° n t ^ le tGSS * The probability of a first return to 

th 

equilibrium occurring at the 2r toss is equal to the probability that 

no return occurs in the first 2r-2 tosses, (p^ Tn ^- nus the probability 

th 

that a return to equilibrium occurs at the 2r toss (p^) • These 
follow from Formulas II. 2, Page 11 and II. 4, Page 11 . As a result we have 

f 2r ’ “2r-2 ‘ *2r (III.A.2) 

In order for exactly 2k-2r of the remaining 2n-2r tosses to be 
positive, a return to equilibrium must occur at the 2n-2k t ^ toss and 
the remaining 2k-2r tosses must be positive. Formulas II. 2 and II. 4 
again tell us that these probabilities are p^ n 2 ^ an< * ^2k 2r 
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»: 



< 








respectively- The total probability for this occurrence is the product 
of the three individual probabilities summed over all possible values 
of r. 



k 

^2n-2k f 2r M '2k-2r = ^2k M '2n-2k 
r=l 



(III. A. 3) 



The second way in which the cumulative number of heads minus tails 
can be positive for exactly 2k tosses up to and including the 2n t * 1 toss 
is for the first 2r tosses to be spent on the negative side of the axis 
and exactly 2k of the remaining 2n-2r tosses to be spent on the positive 
side. Figure 5 illustrates an occurrence of this type. 




toss. This probability was just shown in Formula (III. 2) to be 

^2r = ^2r 2 ~^2r # * For t * ie tota ^ to positive for exactly 2k or the 

remaining 2n-2r tosses, a subsequent return to equilibrium must occur 
in the remaining 2n-2k-2r tosses. These probabilities, as given by 
Formulas (II. 2) and (II. 4), are and p^ n 2 ^ 2r res P ect ^- ve ^y • The 

total probability of this event then becomes 



k 

^2k f 2r M '2n-2k-2r = ^2k U 2n-2k 
r=l 



(III. A. 4) 
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The total probability that up to and including toss 2n of the game, 
the cumulative number of heads minus tails is positive for exactly 2k 
tosses of the coin is given by 



b 2k,2n ^2k ^2n-2k + ^2k ^2n-2k P '2n ^2n-2k* 



(III. A. 5) 



The fourth random variable to be considered is the time (i.e. toss) 
where the cumulative number of heads minus tails first attains the max- 
imum value for the game. This maximum is attained at the 2 k ^ toss if 



S 0 < S 2k’ S 1 < S 2k’ •** S 2k-1 < S 2k 



(III. A. 6) 



S 2k+1 S S 2k* •*** S 2n ^ S 2k (III. A. 7) 



In Figure 6, the first maximum is attained at Point A. This value is 
also attained at Points B and C. 




For 0 < k < n, the event (III. 6) occurs with probability ^l-^k anc * 
the event (III. 7) occurs with probability 2k # For ^ = 0 and k = n, 

these probabilities are p^ and Figures 7 and 8 illustrate these 

two concepts. 
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2 - 



1 - 




Figure 7. The maximum is equal to zero and occurs at the start 
of the game, i.e., k = 0 
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These probabilities again come from Formulas (II. 2) and (II. 4) on 



Page 11. The probability that the first maximum occurs at the 2k 



th 



toss of the coin is therefore equal to 



P '2n-2k* 



(III. A. 8) 



The probability distributions of the last time the cumulative 
number of heads minus tails is equal to zero in the game, of the number 
of tosses on which the cumulative number of heads minus tails is positive, 
and of the toss on which the cumulative number of heads minus tails first 
takes on its maximum value are identical and all are equal to M>2k^2n 2k 



This distribution can be approximated by the Beta distribution with 



parameters cx ^ = % which is more commonly referred to as the Arc 

Sine Distrbution. 



The Arc Sine approximation to the probability distribution of 



|i*2k^2n 2k ev °l ves f rOTn the use of Stirling* s approximation. This 



1 1 




It then 



follows that 




nrr J x(l-x) 



1 



(III.A.9) 



where x = — and 0 < x < 1 



n 



which is the probability density function of the Arc Sine distribution. 



The mean of this distribution is \ and the variance is 1/8. 
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B. UNSYMMETRIC RANDOM WALKS 



Unsymmetric random walks are most easily illustrated by drawing 
analogies to the well known gambler’s ruin problem. In this context, 
two players, each with an initial level of capital and betting unit 
amounts on each play, compete in a game of chance until one of the 
player’s resources are exhausted. Player one represents a gambler with 
limited initial capital resources, z. The second player is referred to 
as the ’’adversary.” His initial resources, a-z, where a denotes the 
total amount of capital in the game, may be either finite or infinite. 
Each play is an independent trial with common probability of success p 
for player one. 

Statistics of interest include the probability of player one being 
ruined, the expected duration of the game, and the expected gain of 
player one in the game. The probability of ruin for player one is 
denoted by and is derived by conditioning on the result of the first 
play of the game. After the first play, the gambler’s capital is z + 1 
with probability p and z-1 with probability q. The resulting dif- 
ference equation is 



q z =pq z+l +qq z-l 1 s z s a • 1 (III.B.l) 

Boundary conditions are q^ = 1 and q a = 0. q^ = 1 states that with 
probability one, player one is ruined when his capital is depleted, 
q =0 states that the probability of player one being ruined when he 

a 

owns all the capital is equal to zero. If p / q, the solution to the 
difference equation is 



= (q/p) a - (q/p) z 
(q/p) a - 1 



(III. B. 2) 
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If p = q, the solution is 



q 



z 




(III. B. 3) 



This derivation follows that given by Feller in Reference 2. 

Playing against an infinitely rich adversary implies that a -> oo # 
In this case 



q 



z 



1 if p ^ q 
(q/p) Z if p > q 



(III. B.4) 



The expected gain in terms of player one’s winnings is then the total 
capital in the game times the probability of player one winning (1-q^) 
minus his initial capital. The expected gain is denoted by G. 



e[g] = a (1 - q z ) -z (III. B. 5) 

The expected duration of the game is again derived by conditioning 

on the outcome of the first event. Let D denote the expected duration 

z 

of a game which begins with player one having capital z. Player one, 
by winning the first play, extends the length of the game by one event 
and a new game with capital z + 1 begins. By losing the first play he 
extends the game by one event and a new game with capital z - 1 begins. 
This leads to the difference equation 



D = p D - + q D « +1 z = 1, 2, ... a - 1 (III.B.6) 

Z Z t I Z “ 1 

For this equation, the boundary conditions are Dq = 0 and = 0. These 
conditions reflect the assumption that the game ends when the capital is 



20 



exausted. When p / q, the expected duration of the game is found to be 



D - _5_ _ 1_-CiZp)1 

z q ' P q " P 1 -(q/p) a 

For p = q, the expected duration of the game is 

D = z (a - z) . 
z 



(III.B.7) 



(III.B.8) 
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IV, SIMULATION OF A RANDOM WALK 



The program simulates a random walk and collects statistics on the 
four variables defined in Section III.A # The first three are the !! last 
return to equilibrium", the number of time units in the trial during 
which the cumulative number of heads minus tails is positive, and the 
point in time during the trial where the maximum gain is first attained, 
given by Formulas III.A.l, 5, and 8 respectively. The fourth statistic 
collected is the cumulative number of heads minus tails which was shown 
to be asymptotically distributed Normal. 

The program also contains a special segment that simulates unsym- 
metric random walks in terms of the gambler f s ruin problem as described 
in Section III.B. Statistics are collected for player one's probability 
of ruin, expected gain in the game, and the expected duration of the 
game. Formulas III.B. 2, 3, 4, 5, 7, and 8 represent the theoretical 
values of the statistics. 

The program is modularly designed so that each statistic collected 
and displayed is a separate subroutine which operates independently 
from the remainder of the program. The only exception is the input- 
output functions which are carried out in a separate module common to 
all subroutines. 

The user interacts with the program via the teletype-writer connected 
to the graphic display unit. Options are selected, parameters are 
specified, and the simulation is carried out entirely under the control 
of the user. The executing time of a particular run may be excessive 
due to the parameters which were specified for the simulation. When the 
possibility of this exists, the user is given the opportunity to 
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re-assign the parameters prior to executing the program. This enables 
the user to avoid execution of a simulation where the parameters have 
been erroneously specified. 

Statistics collected whose empirical distributions are asympototi- 
cally Arc Sine or Normal feature two separate displays. The random 
walk is dynamically displayed as it develops with a frequency plot 
displayed below. The frequency plot is simply an empirical density for 
the random variable being illustrated. This illustrates the develop- 
ment of the probability distribution as it evolves over many repetitions 
of the experiment. The number of events which make up each trial, the 
number of trials, and the number of data cells into which the statis- 
tics will be grouped are all user specified parameters. Figure 9 il- 
lustrates this display. 

The second display repeats the frequency plot. The theoretical 
probability distribution is overlayed and a tabulation of sample and 
theoretical statistics for the expected value and standard deviation 
is made. Figures 10 and 11 illustrate the different displays of sym- 
metric random walks. 

The unsymmetric random walk case illustrated by the gambler’s ruin 
problem displays the random walk as it evolves for each trial of the 
game. Additional axes are drawn representing the initial amount of 
capital possessed by player one and his adversary. The axis repre- 
senting the adversary’s initial capital is the barrier which, when reached 
by the random walk, is interpreted as a victory for player one. The 
equilibrium axis represents the point at which player one’s capital is 
gone and is interpreted as a loss for player one. Theoretical and 
sample values are tabulated and displayed for the probabilities of 
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player one’s ruin as defined by Formulas III.B.2, 3, and 4. His ex- 
pected gain is shown by Formula III.B.5 and the expected duration of 
the game is defined by Formulas III.B.7 and 8. Figure 12 illustrates 
the display created by the gambler's ruin problem. 

This program was written for execution on the XDS-9300 Computer and 
the Adage-10 Graphics Display Terminals, located at the Naval Post- 
graduate School in the Electrical Engineering Laboratory, Spangel Hall, 
Room 500. The programming language used was Scientific Data Systems 
(SDS) FORTRAN IV. Program execution is carried out under the standard 
XDS-9300 operating system and the Program Graphics and Text Editor 
(GATED). The program GATED is a specially developed software interface 
between the XDS-9300 Computer and the ADAGE-10 graphics display ter- 
minals and is not readily compatible with other operating systems. 
References 3 and 4 are technical manuals for SDS FORTRAN IV and GATED, 
respectively. 

No prior knowledge of data processing is required to utilize this 
program. Operation of data processing equipment in the laboratory is 
the responsibility of the individual user, but step-by-step instructions 
are available for all equipment required. During normal working hours, 
laboratory technicians are available to assist the user. 

Instructions for user interaction with the program are an additional 
option which can be accessed from the "menu page" in the same manner as 
any other option. These instructions are not meant to provide any 
theoretical information in support of the simulation presented, but 
serve to delineate acceptable input parameters and to clear up any 
ambiguities which may arise in the user’s first experiences with the 
program. 
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Design features, such as those delineated by Reference 5, are in- 
corporated into the program to make the system as user orientated as 
possible. All input parameters are validated, the user’s needs are 
anticipated wherever they were deemed necessary. Reference 6 lists 
acceptable response times for various situations which arise in user- 
orientated interactive graphics systems. 

Maximum benefits of this simulation can be attained if an instructor 
exposes his class of probability students to the simulation after first 
lecturing on the subject of random walks. As the class lecturers pro- 
gress in the theory, students can experiment with the simulation under 
varying parameters on an individual or group basis. The program then 
complements the material presented in class and reinforces the students 1 
knowledge on the subtleties of the theory. 

The computer program for this simulation is completely annotated 
with comment cards to explain the program. Variables common to the 
entire program are defined in the first segment of the program which is 
the input-output segment. Variables common to individual subroutines 
are defined in their respective routines as appropriate. 

Utilization of interactive computer graphics for instructional pur- 
poses is in its infancy at the Naval Postgraduate School. Professor R. 
Butterworth of the Operations Research and Administrative Sciences 
Department has supervised the development of programs which illustrate 
point processes, Markov chains, queuing models and this thesis on random 
walks. The use of interactive computer graphics is not restricted to 
illustrating probabilistic models alone, but might be useful in illus- 
trating other Operations Research techniques. For example, reliability 
models, war gaming, and regression estimates could be areas for future 
research and development. 
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Figure 9. Graphic display of a random walk and frequency plot. 
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Figure LO. Frequency plot wich Arc Sine distribution overlaved and 
statistics tabulated. 
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Figure 11. Frequency plot with normal distribution overlayed and 
statistics tabulated. 
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Figure 12. Graphic display of the gambler's ruin. 
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